Congrats on getting accepted to NAACL! I thought the Pythia suite could be an interesting family of models to see how scaling affects performance. Eventually I would like add the larger models but need to figure out gpu access before I can do that. The code that I used to calculate the accuracy and sem can be found here https://github.com/samGende/Pythia-lmpubquiz-scaling